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Amendments to the Claims : 

This listing of claims will replace all prior versions, and listings, of claims in the 
application. The amendments find full support in the specification and/or claims as 
originally filed. 

Listing of Claims : 

1. -19. (Cancelled) 

20. (Currently Amended) A method of dynamically determining an optimal 
advertisement to be used by an Internet merchant, comprising: 

(a) receiving configuration data from the Internet merchant, wherein such 
configuration data comprises: 

a sample size of visitors to the Internet website who are to participate in an 
experiment; and 

time-related information concerning the experiment; 

(b) randomly choosing visitors to the Internet website to comprise a sample of 
visitors to participate in the experiment according to the configuration data; 

(c) running the experiment according to the configuration data on the 
randomly chosen sample of visitors to produce sampling data, wherein the experiment 
comprises: 
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presenting a plurality of varied advertisements to different visitors within 
the sample according to the configuration data; and 

measuring the effectiveness of the plurality of varied advertisements on 
the sample; 

(d) dynamically determining an optimal advertisement using real time analysis 
of the sampling data from the experiment[[,]]; 

(e) thereafter using the optimal advertisement determined in step (d); and 

(f) repeating steps (a) - (e) using sampling data obtained in step (c) as 
configuration data in step (a). 

21. (New) An apparatus for determining a currently optimal advertisement for an 
Internet Merchant website through controlled experimentation, the apparatus comprising: 

(1) an eCommerce system; the eCommerce system comprising: 

(a) a webserver; 

(b) an application server configured to perform business logic; and 

(c) a manager console; and 

(2) a dynamic advertising system; the dynamic advertising system comprising: 

(a) a server module, the server module comprising: 
(i) a dynamic sampling engine; and 
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(ii) a logic module; 

wherein, during an experiment, the eCommerce system and the dynamic 
advertising system are configured to: 

(a) receive input experimental parameters at the manager console to 
configure the experiment, wherein the input experimental parameters 
comprise a sampling strategy; 

(b) receive the input experimental parameters at the dynamic sampling 
engine, wherein the dynamic sampling engine produces and outputs a 
plurality of experimental advertisements according to the input 
experimental parameters; 

(c) respond to web page requests received at the webserver from 
customers by passing a sample of the web page requests through the 
application server to the dynamic sampling engine according to the 
input experimental parameters, wherein the dynamic sampling engine 
samples via the webserver randomly distributing web pages 
comprising the plurality of experimental advertisements, one of the 
plurality of experimental advertisements being output per received user 
web page request in the sample of the web page requests; 

(d) utilize the logic module to determine a quantifiable interest for each of 
the plurality of experimental advertisements; and 
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(e) utilize the logic module to determine a currently optimal advertisement 
from the plurality of experimental advertisements via an objective 
function; 

wherein, at the conclusion of the experiment, the currently optimal advertisement 
is returned as a default advertisement in response to subsequent customer web page 
requests. 

22. (New) The apparatus according to claim 21, wherein the quantifiable interest 
comprises a click-through rate; and wherein the sampling strategy comprises different 
forms and formats of an advertisement to be included in the plurality of experimental 
advertisements. 

23. (New) The apparatus according to claim 21 , wherein the currently optimal 
advertisement optimizes a buy-rate. 

24. (New) The apparatus according to claim 21 , wherein the currently optimal 
advertisement optimizes a combination of a click-through rate and a buy-rate; and 
wherein, the currently optimal advertisement is determined by the logic module using a 
weighted formula. 

25. (New) The apparatus according to claim 21 , wherein, at the conclusion of the 
experiment, the currently optimal advertisement is returned as a default advertisement in 
response to subsequent customer web page requests only if a minimum threshold is 
achieved, wherein the minimum threshold is a predetermined percentage better than a 
current default advertisement. 
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26. (New) The apparatus according to claim 21, wherein the web page requests 
received at the webserver from the customers are grouped according to at least one 
variable, the at least one variable being selected from: income group, zip code, 
profession, and buying history; and wherein the experiment is performed on only one 
group. 

27. (New) The apparatus according to claim 2 1 , wherein the web page requests 
received at the webserver from the customers are grouped according to at least one 
variable, the at least one variable being selected from: income group, zip code, 
profession, and buying history; and wherein the experiment is run on all groups. 

28. (New) The apparatus according to claim 27, wherein each group receives a 
different plurality of experimental advertisements. 

29. (New) The apparatus according to claim 27, wherein, at the conclusion of the 
experiment, a currently optimal advertisement for a group is returned as a default 
advertisement in response to subsequent customer web page requests from customers 
similarly grouped. 

30. (New) A method for determining a currently optimal advertisement for an 
Internet merchant website through controlled experimentation, the method comprising: 

(1) conducting an experiment to determine a currently optimal advertisement, the 
experiment comprising: 
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(a) receiving input experimental parameters at a manager console to 
configure an experiment, wherein the input experimental parameters comprise a 
sampling strategy; 

(b) receiving the input experimental parameters at a dynamic sampling 
engine, wherein the dynamic sampling engine produces and outputs a plurality of 
experimental advertisements according to the input experimental parameters; 

(c) responding to web page requests received at a webserver from 
customers by passing a sample of the web page requests through an application 
server to the dynamic sampling engine according to the input experimental 
parameters, wherein the dynamic sampling engine samples via the webserver 
randomly distributing web pages comprising the plurality of experimental 
advertisements, one of the plurality of experimental advertisements being output 
per received user web page request in the sample of requests; 

(d) utilizing a logic module to determine a quantifiable interest for each of 
the plurality of experimental advertisements; and 

(e) utilizing the logic module to determine the currently optimal 
advertisement from the plurality of experimental advertisements via an objective 
function; and 

(2) utilizing the currently optimal advertisement determined in (1) as a default 
advertisement, wherein, at the conclusion of the experiment, the currently optimal 
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advertisement is returned as the default advertisement in response to subsequent customer 
web page requests. 

3 1 . (New) The method according to claim 30, wherein the quantifiable interest 
comprises a click-through rate 

32. (New) The method according to claim 30, wherein the currently optimal 
advertisement optimizes a buy-rate. 

33. (New) The method according to claim 30, wherein the currently optimal 
advertisement optimizes a combination of a click-through rate and a buy-rate; and 
wherein, the currently optimal advertisement is determined by the logic module using a 
weighted formula. 

34. (New) The method according to claim 30, wherein, at the conclusion of the 
experiment, the currently optimal advertisement is returned as a default advertisement in 
response to subsequent customer web page requests only if a minimum threshold is 
achieved, wherein the minimum threshold is a predetermined percentage better than a 
current default advertisement. 

35. (New) The method according to claim 30, wherein the web page requests 
received at the webserver from the customers are grouped according to at least one 
variable, the at least one variable being selected from: income group, zip code, 
profession, and buying history; and wherein experiment is performed only on one group. 
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36. (New) The method according to claim 30, wherein the web page requests 
received at the webserver from the customers are grouped according to at least one 
variable, the at least one variable being selected from: income group, zip code, 
profession, and buying history; and wherein the experiment is run on all groups. 

37. (New) The method according to claim 36, wherein each group receives a 
different plurality of experimental advertisements. 

38. (New) The method according to claim 36, wherein, at the conclusion of the 
experiment, a currently optimal advertisement for a group is returned as a default 
advertisement in response to subsequent customer web page requests from customers 
similarly grouped. 

39. (New) The method according to claim 30, wherein an Internet Merchant 
employee with manager level access can conduct and monitor the experiment via the 
manager console; and 

wherein the web page requests received at a webserver from customers are 
requests for the Internet merchant website. 
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